Update parse start script to handle the case that fsimage txid is cov…#78
Merged
xkrogen merged 3 commits intoFeb 27, 2019
Merged
Conversation
…ered in the range of edits
Contributor
Author
|
@xkrogen Please review this one and thanks in advance! |
Collaborator
|
Hey @fengnanli , thanks for reporting this! I am curious, when will it occur that the edits file does not match the fsimage txn ID? I was under the impression that it always will; I've never seen it differ on our cluster. Maybe there are changes in newer versions of Hadoop which cause this? The code looks good. I have two minor comments:
|
Contributor
Author
|
@xkrogen In our cluster it seems there is never an edit log ending with the fsimage id. I am not sure why, and we are using 2.8.2. |
xkrogen
approved these changes
Feb 27, 2019
xkrogen
left a comment
xkrogen
left a comment
Collaborator
There was a problem hiding this comment.
LGTM! Thanks @fengnanli . Interesting to hear that your edit log txn IDs never align. Maybe a change in 2.8+...
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Sometimes the edit logs doesn't end exactly with the txid from fsimage, so we need to find the proper range which covers this id and thus to pick the right edit log